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ABSTRACT 

The effect of shared information is assessed in terms 
of pilot-controller negotiating behavior and shared 
situation awareness. Pilot goals and situation 
awareness requirements are developed and compared 
against those of air traffic controllers to identify areas cf 
common and competing interest. An exploratory, part- 
task simulator experiment is described which evaluates 
the extent to which shared information may lead pilots 
and controllers to cooperate or compete when 
negotiating route amendments. Results are presented 
which indicate that shared information enhances 
situation awareness and can engender more 

collaborative interaction between pilots and air traffic 
controllers. Furthermore, the value of providing 
controllers with a good-quality weather overlay on their 
plan view displays is demonstrated. Observed 
improvements in situation awareness and separation 
assurance are discussed. 

INTRODUCTION 

The FAA's proposed future National Airspace 
System Architecture [FAA, 1998] calls for expansion of 
existing datalink services to include applications such 
as the Controller-Pilot Data Link Communication 
(CPDLC) system, Automatic Dependent Surveillance 
broadcasts (ADS-B), and Aviation Weather Information 
(AWIN) systems. Such advances will allow 
information which is not uniformly accessible today to 
be shared between pilots, controllers and other users 
(e.g., dispatchers, airport managers, etc.). This sharing 
of information — a digital "party line" — is expected to 
offer several benefits, including the ability to 
communicate graphical information between agents, 
and improved shared situation awareness between 
agents. However, few studies have explicitly 
investigated how shared information may influence 
pilot-controller situation awareness and re-route 
negotiation. 


PILOT AND CONTROLLER GOAL HIERARCHIES 
AND SITUATION AWARENESS 
INFORMATION REQUIREMENTS 

In order to understand the effect of shared 
information in the system and how pilots and 
controllers may act on that information, it was 
necessary first to identify their roles, their motives and 
their informational needs. A comprehensive goal- 
directed task analysis was performed for commercial 
airline pilots [Endsley, et ai, 1998] to complement an 
existing analysis for en-route ATC specialists [Endsley 
& Rodgers, 1994]. Based on extensive focused 
interviews with subject matter experts, each task 
analysis constructed a comprehensive goal hierarchy 
from which the specific situation awareness information 
requirements were derived. 

The individual pilot and controller task analyses 
were compared against one another in order to identify 
areas of common or competing interest between pilots 
and controllers. Figures 1 and 2 depict the high-level 
goals of pilots and controllers, respectively. At these 
higher levels, the goal structures are highly parallel, 
and there is considerable overlap between the two: 

• Assure flight safety 

• Avoid conflicts (e.g., aircraft, terrain, restricted 
airspace) 

• Provide customer service 

• Handle perturbations (e.g., weather, 

emergencies) 

• Manage resources (e.g., people, systems) 

The high-level goal comparison reveals the far- 
reaching effects of re-route decisions. All of the first- 
and second-level goals for both pilots and controllers 
are influenced by the current and fiiture flight path. This 
suggests that re-route negotiations have broad and 
significant ramifications for both pilots and controllers 
and that each should have a vested interest in the 
outcome. 
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Figure 1. Commercial airline pilot top-level goal hierarchy 
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Figure 2. En-route air traffic controller top-level goal hierarchy 


Comparison of the lower-level goals revealed that 
pilots and controllers often have competing interests 
with respect to re-route decisions. For example, pilots 
assess route amendments in terms of time or fuel 
efficiency, whereas controllers assess them in terms of 
their effect on separation and traffic flows. More 
generally, pilots' aircraft-centered goals often conflict 
with controllers* system-centered goals, creating the 
potential for less collaborative negotiations. 

The information upon which such negotiations are 
conducted varies, but pilots and controllers reported 
that traffic and weather information often provide the 
impetus to change path and typically impose 
constraints on the available alternatives. 


SIMULATOR-BASED EXPERIMENT 

Based on the results of the comparative analysis, an 
exploratory experiment was conducted. The experiment 
paired an air transport pilot with an en-route air traffic 
controller in a real-time simulated air traffic 
environment under present-day air traffic control 
procedures. The experiment was directed at re-routing 
situations. In order to limit the number of interacting 
agents, test scenarios focused on tactical routing 
decisions which would preclude the involvement cf 
Airline Operations Centers (AOCs). The identified 
importance of traffic and weather information in re- 
routing situations was reflected in the experiment's use 
of traffic and weather elements in the test scenarios and 
the availability of a traffic and weather datalink as the 
independent variable. 
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Experimental Design 


Table 1. Test Matrix 


Test scenarios were designed to represent complex 
en-route air traffic situations involving convective 
weather and moderate- to high-density traffic flows. 
Weather and traffic hazards were scripted to pose 
routing conflicts to the pilot-controller subject pair. 
The intent was to design conflicts which would play 
on the competing goals of the pilot and controller to 
offer each subject a fairly obvious — yet different — 
solution, thereby raising the need for re-route 
negotiation. The traffic and weather elements were 
designed to create testable responses, a performance- 
based situation awareness probe [Pritchett, Hansman, 
& Johnson, 1996]. Testable response scenarios 
incorporated a hazard element (e.g., an intruder aircraft, 
a weather cell) that required the subject to take action, 
provided s/he was aware of the situation. An 
appropriate action taken by the subject indicated 
situation awareness; inaction indicated a lack of 
situation awareness. Subjects interacted within the 
simulation environment to resolve the traffic and 
weather conflicts. The availability of shared traffic and 
weather displays (via datalink) was manipulated as the 
independent variable as shown in Table 1. Typical 
cockpit and ATC displays are shown in Figures 3 and 
4. 



Figure 3. ATC plan view display with datalink 
enabled. DAL303, positioned inside the six-mile 
radius "J-ring", is the subject pilot's aircraft. The 
weather overlay is six-color NEXRAD imagery. 
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With the datalink disabled, there was no sharing of 
information. Weather information, in the form of six- 
color NEXRAD reflectivity imagery, was available 
only to the subject pilot on the cockpit map display; 
the subject air traffic controller received no weather 
information. Conversely, traffic information — including 
aircraft position, call sign, track, altitude, and ground 
speed — was available only to the subject controller via 
the plan view display; die subject pilot received no 
traffic information. Information was partitioned in this 
way to establish clear information superiority for one 
party relative to the other. 

In the "datalink enabled" configuration, weather and 
traffic information were shared between the pilot and 
controller. The weather information, previously 
available only to the pilot via the cockpit map display, 
was displayed as an overlay on the controller's Plan 
View Display (PVD). Similarly, the traffic information 
was displayed on a prototype Cockpit Display of 
Traffic Information (CDTI). 



Figure 4. Cockpit map display with datalink 
enabled, as seen by the subject pilot of DAL303. 
The integrated CDTI shows proximate traffic co- 
altitude at the 11-, 2- and 3-o'clock positions. 
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Each pilot-controller subject pair performed three 
test scenarios, once with shared traffic and weather 
information (i.e., datalink enabled) and once without 
(i.e., datalink disabled). All scenarios took place in a 
high-altitude sector in Indianapolis Center airspace. 
Each scenario ran for approximately ten minutes and 
featured between 12 and 18 aircraft transitioning the 
sector in the presence of convective weather activity. 

Those aircraft not piloted by the subject pilot were 
controlled by a confederate pseudo-pilot, who also 
interacted with the subject controller and subject pilot 
via radio communication. Certain elements of each 
repeated scenario were changed (e.g., aircraft call signs, 
trajectories of non-factor traffic, etc.) in order to disguise 
the second iteration. 

In order to observe pilot-controller interaction in a 
real-time, complex workload environment, MIT’s 
distributed, interactive, multi-agent simulation facility 
was used [Amonlirdviman, et al ., 1998]. The facility 
was configured to network one part-task advanced 
cockpit simulator, one part-task en-route ATC 
workstation, one multi-aircraft pseudo-pilot station, 
and live voice communications between them, creating 
a real-time interactive air traffic environment. 


Results 

Six pilot-controller teams performed the 
experiment. All controller subjects were Full 
Performance Level (FPL) ATC Specialists with an 
average of 13.3 years of experience, currently working 
the radar position at an Air Route Traffic Control 
Center (ARTCC) in the U.S. All pilot subjects were 
jet transport rated pilots with an average of 10,117 
hours. 

Situation Awareness . Figure 5 summarizes the 
results of the traffic-related testable response probes. 
Pilots, without the benefit of a traffic display in the 
non-datal inked configuration, did not demonstrate 
awareness of any of the traffic-related testable response 
conditions. When provided a shared traffic display, 
pilots demonstrated awareness of 56% of the traffic- 
related testable response conditions. In some cases, the 
controller recognized the traffic conflict before it became 
a significant threat to the pilot and either advised the 
pilot of the traffic or vectored the pilot accordingly. In 
such cases, the pilot’s opportunity to independently 
recognize and respond to the hazard was precluded, and 
the testable response result for the pilot therefore was 
labeled "ambiguous". 


Pilot and controller situation awareness was 
measured using the testable response method. Aircraft 
state and trajectory data were digitally recorded at 20 
Hz. Radio communications were digitally recorded and 
coded using a methodology adapted from Foushee, 
Lauber, Baetge, & Acomb {1986]. Workload 
measurements were taken using the NASA Task Load 
Index (NASA-TLX) [Hart & Staveland, 1988]. 
Subjective ratings regarding the value of the shared 
information were also collected. 


Controllers demonstrated a high level of awareness 
of the traffic-related testable response conditions. In 
some cases, a deviation requested by the subject pilot 
resolved the traffic-related testable response condition 
before it arose; such cases were labeled "ambiguous" 
with respect to controller situation awareness. 

Figure 6 summarizes the results of the weather- 
related testable response probes. Pilots, having the 
benefit of the weather display for all test scenarios, 
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Figure 5. Pilot and controller awareness of 
traffic- related testable response conditions 
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Figure 6. Pilot and controller awareness of 
weather-related testable response condition 
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demonstrated awareness of all of the weather-related 
testable response conditions. Controllers, without the 
benefit of a weather display in the non-dataiinked 
configuration, demonstrated awareness of only 50% of 
the weather-related testable response conditions. When 
provided a shared weather display, controllers 
demonstrated awareness of 94% of the weather-related 
testable response conditions. 

These results indicate that pilot situation awareness 
with respect to traffic improved with the addition of a 
CDTI. Similarly, the results suggest that controller 
situation awareness with respect to weather improved 
with the addition of a weather overlay to the plan view 
display. These results confirm that shared information 
via air-ground datalink can improve situation 
awareness for both pilots and controllers. 

Controllers’ Weather Awareness . En-route 
controllers report that the the weather information 
currently provided on their plan view displays is of 
limited value. To compensate, they use pilot reports 
(PfREPs), aircraft trajectories, and pilot requests to 
construct a mental picture of the areas affected by 
weather and to project how the traffic flow will be 
affected. To gain some insight into the accuracy of this 
heuristic, controllers were asked to perform a simple 
recall task at the conclusion of each non-datalinked 
scenario. Each controller was asked to indicate, on a 
blank sector map, the size and location of the weather 
cell(s) as inferred from the aircraft trajectories and voice 
communications from pilots. Figure 7 is a sample of 
the results. To first order, the aircraft trajectories tend 
to wind around the regions drawn by the controllers. 
This is consistent with the strategy controllers report 
using to infer the location of weather in their sector. 
Controllers’ skill at inferring the location of weather 
using this heuristic was found to vary widely, both 
within and between subjects. 



Figure 7. Example of controller’s inferred weather 
location (black). Aircraft tracks (gray) and weather 
image have been overlaid for reference. 


Loss of Separation Events . In the 36 test scenarios, 
five loss of separation events' were observed, all of 
which occurred with the datalink disabled. It is 
important to note that the test scenarios were 
challenging by design. Controllers were operating an 
air traffic sector other than their usual "home" sector 
and did not have the benefit of a conflict alert function 
or a D-side controller to assist them. However, the fact 
that every loss of separation occurred in the non- 
datalinked environment does suggest that shared 
information may help controllers build and maintain 
situation awareness with regard to separation issues. 

Communication and Negotiation . All radio 
communication was recorded and coded by category 
and topic. With the datalink enabled, the pilot and 
controller made more voluntary suggestions to one 
another for specific route amendments. This verbal 
exchange of re-routing ideas, options and preferences 
was rarely evident when the datalink was disabled. 
This result is marginally significant at the 91% 
confidence level (p < 0.09). In addition, controllers 
were more proactive in providing weather advisories to 
pilots when they had the weather information overlay. 
This result is statistically significant at the 99% 
confidence level (p < 0.01). 

Workload . Pilot and controller workload was 
measured using the NASA-TLX. In general, the 
availability of shared information did not affect the 
workload in any systemic way, either individually or 
in a team sense. 

Subjective Responses . At the conclusion of each 
test session, subjects were asked to provide a 
subjective rating of the value of the shared information 
on a scale ranging from "very detrimental" to "neutral" 
to "very valuable". Pilot feedback was unanimously 
favorable, and all six of the controllers rated the shared 
weather information as "very valuable". While 
controllers were enthusiastic in their support for the 
shared weather display, their opinions on sharing their 
traffic information with the cockpit were mixed. Some 
controllers suggested that it could be useful to 
controllers and pilots when sequencing aircraft in the 
terminal area. Others expressed concern that arming 
pilots with such information might make pilots "less 
complacent" with regard to their approved clearances or 
assigned vectors. 


1 A loss of separation is defined as lateral separation of 
less than five miles and vertical separation of less than 
1000 feet. 
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CONCLUSIONS 
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It is generally expected that by sharing information 
between pilots and controllers, situation awareness will 
be improved on either side. With improved situation 
awareness, more collaboration between the two parties 
is anticipated. Such collaboration is expected to lead to 
improved performance on an individual and system- 
wide basis. 

The results of this study tend to corroborate these 
expectations. By sharing traffic and weather 
information, pilots’ and controllers’ situation 
awareness with respect to traffic and weather was 
improved. Sharing of this information led to more 
collaborative interaction, as evidenced by more frequent 
advisories from ATC and the unsolicited exchange of 
suggestions for alternative, more favorable routings. 
With improved situation awareness and increased 
air-ground cooperation, safety was improved, as 
evidenced by the lack of separation violations in the 
datalinked configuration. 

The availability of a N EX RAD weather overlay 
clearly benefited the controllers and the control system 
in general. Without the weather overlay, controllers 
had a difficult time anticipating the effects of weather on 
the traffic flow. As a result, controllers were faced with 
a high number of tactical deviations requiring time- 
critical conflict management. Attention to these 
immediate-term situations generally came at the 
expense of longer-term strategic planning. Furthermore, 
without good situation awareness regarding the 
location of weather-impacted areas, the controllers’ 
primary conflict resolution strategy was simply to meet 
the pilots’ re-route requests wherever possible. 
However, as suggested by the situation awareness 
analysis, the pilots’ requests typically reflected a 
desire to select the most efficient route that would 
avoid the weather; the impact of said route on the 
broader traffic flow was not an apparent goal of pilots. 
Thus, in attempting to honor pilots’ re-route requests, 
controllers were in effect subordinating their own goal 
of maintaining an orderly traffic flow to the pilots’ goal 
of selecting an efficient route. Ultimately, several 
separation violations occurred. When the weather 
overlay was provided, controllers were better able to 
anticipate routing constraints, enabling them to shift 
their attentions from tactical control to strategic 
planning. No separation viloations occurred in this 
configuration. The markedly improved performance (in 
terms of separation assurance and situation awareness) 
and strong subjective preference for the weather display 
suggests that NEXRAD-type weather information 
should be made available on the PVD. 
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